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Abstract 

Recently, Taneja [7J studied two one parameter generalizations of J -divergence, 
Jensen-Shannon divergence and Arithmetic- Geometric divergence. These two gen- 
eralizations in particular contain measures like: Hellinger discrimination, symmetric 
chi-square divergence, and triangular discrimination. These measures are well known 
in the literature of Statistics and Information theory. In this paper our aim is to prove 
metric space properties for square root of these two symmetric generalized divergence 
measures. 
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1 Introduction 
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Let 



P = (Pl,P2, -,Pn) 



Pi > O^Pi = 1 > , n > 2, 



i=i 



be the set of all complete finite discrete probability distributions. For all P, Q G T n , let us 
consider two generalized symmetric divergence measures. These measures are well known in 
the literature on information theory and statistics. 

Let us consider the measure 
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for all P, Q G T n 

The measure (II. ip was studied for the first time by Taneja |6J and is called general- 
ized symmetric arithmetic and geometric mean divergence. The measure (ll.ip admits the 
following particular cases: 

(i) e-i(P||Q) = jA(p||g). 

(ii) £ 1 {P\\Q) = I(P\\Q). 

(iii) £ 1/2 (P||Q)=4i(P||Q). 

(iv) UP\\Q) = T(P\\Q). 

(v) 6(p||Q) = i*(p||g). 



where 



^ (Pi 



and 



A( p||Q )= £M^_JL 

i=l Pl + ^ 



n , 

d(piiQ)=i-x;( 

i=l ^ 
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Pi + ft 



are the triangular discrimination and d-divergence respectively. The measures I(P\\Q) and 
T(P||Q) are the well-known Jensen- Shannon divergence [5J [T] and Arithmetic- Geometric 
mean divergence [Bj, respectively. 



Let us consider now the other measure 



UP\\Q) 



for all P, Q G T r 



J S (P||Q) = [ S ( S -1)]- 1 



E(pfrf"*+Pf"*9f)-2 



i=l 



8^0,1 

, , (1-2) 
WIG) = E (ft -ft) In (a), * = 0,1 



The measure ( ll.2p can be seen in Burbea and Rao pQ and Taneja [7]. The expression 
( II. 2p admits the following particular cases: 

(i) ^ 1 (P\\Q) = ( 2 (P\\Q) = ly(P\\Q), 

(ii) ( (P\\Q) = UPWQ) = J(P\\Q), 

(iii) ( 1/2 (P\\Q)=8h(P\\Q), 

where 

(Pi - Qi) 2 (Pi + ft) 



*( J P||Q) = x 2 (^IIQ) + x 2 (g||P) = E 
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and 



1 n 

h(P\\Q) = 1 - B(P\\Q) = ~J2 (VPi ~ V® 



;i.3) 



i=i 



are the symmetric x 2 - divergence and Hellinger's discrimination respectively. The measure 
J(P\\Q) is the well-known J-divergence [I]. For detailed study of the measures ( II .ip and 
(OD refer to Taneja [7]. 



The symmetric divergence measures ( ll.ip and (II. 2p admit several particular cases. An 
inequality among these measures jH] is given by 

^A(PHQ) < I{P\\Q) < h(P\\Q) < Ad(P\\Q) < ±J(P\\Q) < T(P\\Q) < ±9(P\\Q). (1.4) 

An improvement over the inequalities given in (ll.4p can be seen in Taneja [8j [9]. 

In this paper our aim is to prove metric space properties of the square root of the measures 

( rrrp and (0|) . 

2 Generalized Divergence Measures and Metric Spaces 

In this section we shall prove the metric space property of the square root of the measures 
given in (JT7TJ) and (Oj) . 



2.1 JS and AG — Divergences of Type s 

Let the function ( s (p, q) '■ R + X R- + — > R + be defined as 



I(p, g ) = § In + S = 

s = 1 



Cs(p,q) = < 



T(p, = In 
In view of (12.11). we can write 



p±q 



(2.1) 



Cs(P\\Q) = ^2Cs(pi^), 



i=l 



for all P, Q E r n 



(2.2) 



Theorem 2.1. T/ie measure given by a/ Cs(p> <?) a metric over R + . 
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Proof, (i) Initially we shall prove the result for s ^ 0, 1. It is sufficient to show the triangle 
inequality: 



y/L a (p,q) < ^L s {p,r) + ^L s {r,q), \fp,q,reR + , s ^ 0, 1 (2.3) 

Let us write 

K pq (r) = y/L 8 (p,r) + y/L 8 (r, q). 

Now we shall prove that K pq has only one minimum at r = p = q. The derivative o iK pg 
with respect to r is 

K' (r) = L ' 8 ^ + L ' s ^ g) 
mU 2 V ^7) + 2 V ^M)' 

where 

L' s (p,r) = ^L s (p,r) 

(1 - s)r- s (p + r) s + (p 1 ^ + r 1_s )s(p + r)^ 1 - 2 s 
~ s(s- 1)2 S +! 

(1 - s)(l + t) s + s(£ 1_s + l)(i + I) 5 " 1 - 2 s 
f=t s(s - 1)2 S+1 ' 

Also, we can write 

V L s(P,r) = y/fy/L a (t,l), 

r 

Multiply by 2^ and define the function h (t) by setting 

^L' s (p,r) n Ls (t) 

where 
and 



Thus the sign of hL a (t) depends only on the sign of n Ls (t). We have 

< (*) = - (i ^r 2 K 1 n + (i + 1 1 - 3 )} . 

This give n' Ls (t) < 0, V£ > and Vs =>- n Ls (t) decreases monotonically in (0, +oo). 

As /il s (1) = 0, riL s (t) changes the sign at t — 1, therefore tiL a (t) changes the sign at t — 1. 
This gives 

, , x f > 0, £ < 1 
< 0, t > 1 
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for any s. 

As 2 = t, then 2 = IE = Bt, where 8 = 2. Therefore, 

dr 

Now, suppose 8 > 1 (q > p), this give: 

• for t < -5: hL s {t) and /iL s (/?t) have the same sign + 

• for t > 1: h Ls (t) and h Ls {j3t) have the same sign - 

• for t e (± l) t/3 > 1 =>■ h L ,(tp) < 

• for t G =>h L .(t) >0 

Finally, we have for t G (|>l); > e h Ls (t8) < 0. Since \h Ls (t)\ > \h Lg (t(3)\ (h La is 

monotonically decreasing), then h Ls {t) + liL a {f3t) > 0. For t > 1, h Ls {t) < e h Ls {tf3) < 
and h Ls (t) + h Ls (8t) < 0. 

Therefore, dK ^ B indeed changes from positive to negative sign at t — 1 (r = p) so that 
there is a minimum at t — 1. Now, we shall show that this happens only once. 

Since hi s is monotonically decreasing this implies that h' L < and we know that the function 
hj Js changes the sign only once. This gives 

j t (h Ls (t) + h La (tB)) = h' Ls (t) + h' Ls (tB) < 0, 

The case q < p can be investigated in a similar fashion. Symmetry of L s allows us to take 
t = 2 and - = Bt with 8 = - > 1. From this we conclude that there is a minimum at r = q. 

r r 1 1 q 

Repeating the same process by substituting t : = 2 and - = fit with 8 = - we conclude that 
the function K' also changes the sign at r — q. This proves the result (12. 3p for s ^ 0, 1. 

(ii) Now we shall prove the result for s = 1. We have to show that 

a/T(p, q) < y/T(p, r) + y/T(r, q), Vp,g,reR + . 

Let us write 

T pq( r ) = \fT{P^ r ) + VT(S, q), 

then obviously, 

T , ^ _ T'(p,r) i T'(r,q) 



2y/T(p,r) 2y/T(r,q)' 
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Now, we have 



T'(p,r) = ±T(p,r) 



This give 



where 



2 11 \2y/pr) + V 2 J dr 

1 / p + r \ r -p 

2 11 VSv^y 4r 



T'(p,r) M*) 



In 



p + r 
2y/pr 



7 32r 



21n (^)+( 1 " t ) 



M*) : 

Let us take now ^ — jit where j3 = |, then we can write 

v^Z^(r)| =M*) + M#)- 

r 

Let us study now the function hx{t). Call Ht{x) and rfy (£) the functions in the numerator 
and denominator of hrif), respectively. Since we know that dr{t) > 0, Vr > 0, then the 
sign of h T {t) is determined by n T (t). 



Now, 



n' T (t) = 2 



Vt d ft + 1 



(t + 1) dt V 



;i + t 2 



From this we conclude that is decreasing Vt > and n' T {t) ^ 0, Vt G R. Since 

= then n T (t) changes the sign at t — 1. This gives that /it(^) changes the sign at 

t = I. 

Thus we conclude that hx(t) is decreasing function of x and 

, , , f > 0, * < 1 
hr(t) < 

\< 0, t>l 

Set j3 > 1. In this case, 

• for t < /i-r(^) and h T (j3t) have the same sign + 

• for t > 1: /i-r(^) and h T (/3t) have the same sign - 
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• for t G (jj, 1 ) tp > 1 h T (tp) < 

• for t G (±, I) => h T (t) > 

Finally, for t G (jj, l\, hr(t) > e hr{t/3) < 0. Thus we observe that the sign of hr{t) + 
hr{/3t) may change in ( -|, 1J. Now, we shall show that this happens only once. 



Since h' T is monotonically decreasing this implies that h' T < and we know that the function 
/^changes the sign only once. This gives 

j t (h T (t) + h T (t(3)) = h' T {t) + h' T (t(3) < 0, 

Repeating the same process by substituting t :— % and *j = (3t with — 2 we conclude that 
the function T' also changes the sign at r = q. 

(iii) For s = the result is already given in Endres and Schindelin [3]. □ 

2.2 J — Divergences of Type s 

Let the function £ s (p, q) : R + x R + — » R + be defined as 

, fJ.(j>,i) = [»(»- I)]"' W+P 1 "^ -(? + «)], 

&ta,) = i /&>.«) = &> . = o,i (24) 



In view of (12. 4p , we can write 



for all P, Q G T n 

Theorem 2.2. TTie measure given by a/£ s (p, q) is a metric space over R + . 

Proof, (i) Initially we shall prove the result for s ^ 0,1. It is sufficient to show the triangle 

inequality: 

y/Js{p,q) < VJs(P,r) + V-Ur,q), Vp,g,rGR + . (2.6) 

Let us write 

F Pq (r) = ^J s (p,r) + y/J a (r, q), p ^ q, 

then obviously, 



W J s(P,r) 2y/J a (r,q)' 
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Now, we have 



J' s (P,r) = -^ J s(P^ r ) 



(1 — s)p s r s + sp s r s — 1 

s(s-l) 
t s r + t 1 ~ s r — tr — r 
=t s(s — 1) 



Also, we can write 



Let us write 



where 



and 



p=rt 



Thus the sign of hj a (t) depends on the sign of nj a (t). 

n , J .(*) = -f- 1 -r-. 

Thus n'j (t) < 0, Vt > and Vs G R — {0, 1} =4> n Js (t)is decreasing Vi > 



As hj a (l) = 0, n Js (t) changes the sign at t = 1 and therefore /ij s (t) changes the sign ait — 1. 
Thus for any s, we have 

'> 0, i < 1 
<0, i>l 



mo 



As 2 = t, then 2 = IE = fit, where = Therefore, 

V~r^ = h Js {t) + h Ja m. 

Now, 

• for t < jj-. h Js (t) and hj a (/3t) have the same sign + 

• for t > 1: hj a (t) and hj a ((3t) have the same sign - 

• for t G l) tp > 1 h Ja (tf3) < 
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• for t G =>hj.{t) >0 

Finally, for t G Q^, lj, /ij 3 (t) > e hj s (t(3) < 0. Thus we observe that the sign of hj s (t) + 
hj 3 (/3t) may change in ( 4, 1 ) . Now, we shall show that this happens only once. 



Since h'j is monotonically decreasing this implies that h' Js < and we know that the function 
hj a changes the sign only once. This gives 

j f (h Js (t) + h Jm (jtfi)) = ti Js (t) + h' Js (t/3) < 0. 

Repeating the same process by substituting t :— * and 2 = f3t with — | we conclude that 
the function T' vq also changes the sign at r = q. 

(ii) For s = 0, 1, the result follows by the continuity of the function £ s (p, q) with respect to 
s. □ 

3 Asymptotic Approximation 

In this section we shall bring asymptotic approximation of the measures given by (II. ip and 
(II. 2p . For this, first we shall give a general result for Csiszar's /-divergence then the other 
cases become as particular. 

Given a function/ : (0, oo) —> R, the f-divergence measure introduced by Csiszar's j2] is 
given by 



C/(P||Q) = f>/ (t)' 



(3.1) 



for all F, Q G T n . 

The following result is well known in the literature [2]. 

Result 3.1. If the function f is convex and normalized, i.e., /(I) = 0, then the f-divergence, 
Cf(P\\Q) is nonnegative and convex in the pair of probability distribution (P,Q) G T n x T n . 

Based on Result 3.1, we can prove some properties of the measures (jl.ip and (II. 2p . 

Theorem 3.1. If f is twice differentiate at x = land /"(I) > 0. Also /(l) = ; then 

C f (P\\Q)*^p-x 2 (P\\Q)- (3.2) 



Equivalently, 



x\P\\Q) 2 asF ^V- 
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Proof. From Taylor's series expansion, we have 

/"(I) 



/(*) = f'(l)(x - 1) + J -^{x - If + k(x)(x - l) 2 , 



where /(l) = and k(x) — > — > 1. Hence 

'ft 



Qif 



/'(Dfe - *) + m<i^ + * f fez*)!. 

2 $ % 



Approximating pi — >■ g 4 and summing over alH = 1, 2, ....n we get the required result. □ 
Proposition 3.1. TTie following results hold: 

(i) ( s (P\\Q)^lx 2 (P\\Q),VseR. 

(ii) UP\\Q)^x 2 (P\\Q),^seR. 

Proof, (i) For all x > and s G (—00, 00), let us consider 



^s(x) 



flnx-(^i)ln(^i), 
(^) ^ (fj§ 



s = 
s = 1 



(3.3) 



in (13TTp . then we have C>(P||g) = C(^IIQ), where t.(P\\Q) is as given by (O]) . 
We have 



1 - x" 1 - In a; - 2 In (^) , 



s-l 



s = 

s = 1 



and 



This gives 



ip"(x) 



X 



-s-l 



x + 1 



(1) = 4- 



(3.4) 
(3.5) 



Expression (13. 2p together with (13. 4p and ( 13. 5 p completes the proof of part (i). 
In particular, when s = in ( 13. 3 j) the result is obtained in Endres and Schindelin [3]. 
(ii) For all x > and s G (—00, 00), let us consider 



4>s{x) 



[s(s - 1)] 1 [x s + x 1 - 5 - (1 + x)} , s ^ 0, 1 
(x — 1) lnx, s = 0, 1 



(3.6) 
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in (EEJ, then we have C f (P\\Q) = £ a (P\\Q), where £ a (P\\Q) is given by (TOj) . 



We have 

[s(a - [s^- 1 + x~ s ) + x~ s - 1] , s ^ 0, 1 
1 — + lnx, s = 0, 1 



0' s (x) 
and 



i 



(jj' s (x) = x s - 2 + x- s ~ l . (3.7) 

This gives 

(1) = 2. (3.8) 

Expression (13. 8p together with (13 .ip and (13.21) completes the proof of part (ii). □ 
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